Using GitHub Action for CICD and Integrating Git Secrets

Posted on Views: Waline: Word count in article: 832 Reading time ≈ 3 mins.
The author records the process of migrating the project's CICD pipeline from Jenkins to GitHub Action, shares relevant tutorial links, elaborates on the environment, triggering conditions, container usage, etc., and also provides the yml configuration file and precautions.

In the past two days, I migrated the CICD pipeline of a project from Jenkins to GitHub Action. I encountered some problems during the process and made a record.
First, here are some good tutorial links I found in these two days:

  1. The first one is a tutorial written by the great Ruan Yifeng in 2019. It’s easy to understand. As expected of a master, he started writing tutorials when Action just came out.
    Github Actions 入门教程

  2. The second is the official GitHub workflow syntax tutorial. It is recommended to read the table of contents thoroughly and then find the parts you need for detailed reading.
    Workflow syntax for Github Actions

  3. The third is the official repo of git secrets. Git secrets is a tool that can be used to scan whether the code contains sensitive information such as passwords and usernames.
    git secrets

Instructions:

  1. Read the instructions in combination with the yml file below and carefully check the comments in the yml file. I wrote a lot of information in the comments.
  2. Our environment is GitHub Enterprise Edition, with a self - hosted runner server, which is different from github.com.
  3. We adopted a relatively common way to trigger the CICD job: when a pull request is initiated on a branch, the CICD for the test environment will be triggered; when code is merged into the master branch, the CICD for the production environment will be triggered.
  4. I chose to place the actual running process of the pipeline in a container because in this way, all dependencies can be encapsulated into the container. Even if the runner server is replaced in the future, as long as Docker can be run, compilation can be carried out.
  5. Important!!! Since I configured the pattern of git secrets in the workflow configuration file (the yml file below) (that is, the git secrets --add xxxxx command), I need to configure git secrets not to detect this yml file (git secrets --add --allowed .github/workflows/main.yml:.*). Otherwise, git secrets will always prompt that there is private information in the code, and the pipeline will fail.
  6. Important!!! The configuration information of git secrets is in the .git directory of the repo. So remember to delete this repo every time before running. Otherwise, when executing git secrets --scan, it will fail. rm -rf * cannot delete hidden directories, so you need to execute rm -rf .git separately. Points 4 and 5 cost me nearly a day…
  7. I used the if syntax and branch name conditions to determine whether it should be deployed to the test environment or the production environment.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
name: CICD

# Conditions for triggering the pipeline
on:
  push:
    branches: [ master ]
  pull_request:
    branches: [ master ]

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
  # This workflow contains a single job called "build"
  build:
    # runs-on is followed by the tag of the server where the runner is located
    runs-on: [ myTag ]
    # The configuration under container is used to start a Docker on the above server as the actual execution environment of the pipeline
    # If a public image from Docker Hub is used here, no credentials are needed. If it is a private repository, configuration is required.
    container:
      image: xxxxxxxxxxxxxxxxxxxxxxx
      credentials:
        # ${{secrets.xxxxxxxxxxx}} makes use of GitHub's secrets feature. You can add the required
        # keys, tokens, and other information in the repo's setting-->Secrets, and then use them in the workflow in the form of ${{secrets.xxxxxxxxxxx}}. This is much better than Jenkins.
        username: ${{secrets.xxxxxxxxxxx}}
        password: ${{secrets.xxxxxxxxxx}}
      env:
       # Set environment variables in the container
        AAA: BBB

    # Steps represent a sequence of tasks that will be executed as part of the job
    steps:
      # Clean up the environment before starting editing.
      # rm -rf * cannot delete hidden directories, so .git needs to be deleted separately because the configuration files related to git secrets are all in it.
      # If not deleted, when executing the git secrets --scan command, the shell exit code will return 1, and the action's workflow will fail.
      # I spent a day on this...
      - name: Initial Env
        run: |
          rm -rf *
          rm -rf .git
          ls -a
          
      # Use the official action to pull down the code
      - uses: actions/checkout@v2

      - name: Secrets Scan
        run: |
          git secrets --install -f
          git secrets --register-aws
          git secrets --add --allowed .github/workflows/main.yml:.*
          git secrets --add xxxxxxxxxxx
          git secrets --scan

      - name: Stg Build Image
        if: ${{github.ref != 'refs/heads/master'}}
        run: |
          Stg Build Command
      
      # Use the if syntax and branch name conditions to determine whether it should be deployed to the test environment or the production environment
      - name: Stg Deploy
        if: ${{github.ref != 'refs/heads/master'}}
        run: |
          Stg Deploy Command
          
      - name: Prod Build Image
        if: ${{github.ref == 'refs/heads/master'}}
        run: |
          Prod Build Command
          
      - name: Prod Deploy
        if: ${{github.ref == 'refs/heads/master'}}
        run: |
          Prod Build Command
       
      # Remember to clean up the environment at the end to avoid information leakage as much as possible
      - name: Clean up
        run: |
          rm -rf *
          rm -rf .git